1. Field of the Invention
This invention pertains generally to methods and systems for removing artifacts found in block-encoded video signal, and more particularly to removing blocking artifacts and contouring effects found in block-encoded video signals.
2. Description of Related Art
The popularity of digital video has increased dramatically. In order to minimize the bandwidth for transmission and the amount of storage space, compression techniques are utilized to reduce the size of video. Video is a sequence of frames of image data. Each frame is typically displayed as a two-dimensional array of pixels. Each frame is subdivided into data subsets which we denote as “macroblocks” of data. Motion estimation is performed on the macroblock to find the best match between the macroblock and the reference frame. The residual error is then transformed by discrete cosine transform (DCT) and quantized.
For each macroblock, motion estimation is performed by finding the candidate which can most efficiently represent the macroblock in the reference picture within the search range. This best-matched location is encoded in the form of a motion vector. At the decoder side, the motion vector is then used to reconstruct the video frame by copying the pixel values at the corresponding location in the reference frame. Because motion compensation can reduce temporal redundancy, it is widely used in current video standards (e.g., MPEG and H.263) to achieve the goal of compression.
When a transform coded image is reconstructed, the image may include a number of visible artifacts of the compression process, e.g., “blocking effect”, “contour effect,” “ringing artifact,” “staircase noise,” etc.
The blocking effect is the result of the approximation of the DC transform coefficient during quantization. In quantizing the transform coefficients the encoder selects one of the available quantization parameters closest in value to the coefficient to represent the coefficient. This “rounding off” can produce pixels of visually different colors in adjacent blocks even though the corresponding pixels of the original image were nearly identical. As a result, blocks of pixels created for the transform coding process may be visible in the reconstructed image. The blocking effect becomes more pronounced as the quantization steps are coarsened to reduce the data rate. The blocking effect is a particular problem along edges or color boundaries in the image or in areas of the image that are of relatively uniform color.
Generally, in a low bit-rate case, the blocking noise will occur at the block boundaries and the motion compensation scheme will make things worse by propagating the blocking noise to the following predicted frames. Various de-blocking techniques have been proposed for reducing the blocking noise. These techniques can reduce the blocking noise near the block boundary, but fail to reduce the noise at the center region of the block. The noise at the center region of the block is due to the blocking noise from the previous frame which is used as the reference frame. After motion compensation, the noise at the boundary from the previous frame is moved to the center of the block and existing de-blocking techniques fail to detect this situation and noise is unable to be reduced.
FIG. 1 shows the diagram of a typical video decoding system 10. Examples of this system are DVD players or direct satellite receivers which use MPEG-1, 2, 4 as the video coding standards. The system receives the video bitstream 12 and decodes the bitstream with a variable length decoder (VLD) 14. If the macroblock is intra-coded, the quantized coefficients are inverse quantized (IQ) followed by inverse DCT (IDCT) 16 to get the reconstructed pixel values. If the macroblock is inter-coded, the decoded pixel value will be the sum of the output of IDCT 16 and the motion-compensated references using motion vector (MV) values 18 (the loop shown in FIG. 1). The dashed-line in FIG. 1 shows a regular decoder 20 which requires the conformance to the video standard. The decoded video frame is processed by the de-blocking filter 22 to reduce the blocking noise and the modified video is then output to display 26.
The implementation of the de-blocking filter as shown in FIG. 1 is not normative in the MPEG-1, 2, 4 standard, because the system has the tendency to have high computation requirement or be unsuccessful in reducing the noise at the center region of a block. A sample filtered frame 29 using the system of FIG. 1 is shown in FIG. 2. By decoding the information contained in the bitstream and the previous decoded frame stored in the frame memory 25, we can reconstruct the frame 28, which is the output of the regular decoder 20. There are noticeable blocking noises on the reconstructed frame and de-blocking filter is necessary to reduce the noise. However, most of the filters are implemented only to check the pixels close to the block boundary 32, which will fail to reduce the noise caused by the noise propagated by motion compensation. As FIG. 2 shows, the blocking noise is still observed at the center region of the block 30 after de-blocking filtering.
In addition to blocking artifacts, contouring effects will show in the region with a smooth gradation in the luminance level in low bit rate cases. An example of contouring effect is shown in FIGS. 3A and 3B. FIG. 3A illustrates a scene of a smooth surface that is lighted by a lamp from a tilted angle. In the upper middle part of FIG. 3B, groups of neighboring pixels have the same values after decoding. The boundaries of these constant pixel regions are called “contours,” and are generally annoying to the human visual system.
Accordingly, it is an object of the present invention to provide an improved system for filtering blocking noise from a blocked video image.
It is a further object of the present invention to provide an improved system for reducing contouring effects from a blocked video image.
At least some of these objectives will be met in the inventions described hereinafter.